Frictionless inquiry processing

ABSTRACT

Images capture an object that a person in a frictionless store is looking at. A user interface is automatically initiated on a device identified as the object. If the person is holding an item, the user interface is automatically placed in a state that displays the item details and pricing within the user interface and provides a link to all item descriptions in possession of the user within the frictionless store along with a running price total of all the items. If the person is not holding any item, the user interface is automatically placed in a state that displays item details and pricing within the user interface to all item descriptions in possession of the user within the frictionless store along with a running price of all items.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.16/368,422, filed Mar. 28, 2019, which application and publication isincorporated herein by reference in its entirety.

BACKGROUND

Recent advancements in image processing has permitted retailers toprovide a frictionless shopping experience to its consumers. In africtionless store, an individual is allowed to check in with a storeelectronically, browse for items to purchase, place items in bags/carts,and exit the store without any interaction with a store agent or with acheckout station. The items that the individual possesses when leavingthe store are recognized through image processing and an accountassociated with the individual is automatically charged for the price ofthe items. This allows for quick and easy shopping and is referred to asa frictionless store or frictionless shopping because the consumer doesnot have to interact with any agent of the store or any terminal of thestore to purchase items.

However, as experience grows with frictionless stores it is apparentthat often the consumers want to have some way of rapidly telling arunning price total and listing of goods that the consumers have pickedup while in the frictionless store and want some mechanism foridentifying a price for a given item when such information is missing ornot readily available from signs in the store. Scan terminals mayprovide a price of a given item but these terminals may experiencequeues of consumers trying to discover the prices of their goods andqueues defeat the very purposes of a frictionless experience. Moreover,the scan terminals do not provide an interface by which the consumer canidentify a current listing of the goods possessed by the consumer and arunning price total for all of those goods.

Thus, there is a need for frictionless inquiry techniques that allowconsumers to identify item prices and obtain running item details withpricing totals within a frictionless store.

SUMMARY

In various embodiments, methods and a system for frictionless inquiryprocessing are presented.

According to an embodiment, a method for frictionless inquiry processingis presented. A consumer is identified within an image as looking at adevice. A user interface is caused to be automatically initiated on thedevice based on the consumer being identified as looking at that device.The user interface is instructed to display on the device informationassociated with at least one item that is in a possession of the user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of a system for frictionless inquiry processing,according to an example embodiment.

FIG. 2 is a diagram of a method for frictionless inquiry processing,according to an example embodiment.

FIG. 3 is a diagram of another method for frictionless inquiryprocessing, according to an example embodiment.

FIG. 4 is a diagram of a system for frictionless inquiry processing,according to an example embodiment.

DETAILED DESCRIPTION

FIG. 1 is a diagram of a system 100 for frictionless inquiry processing,according to an example embodiment, according to an example embodiment.It is to be noted that the components are shown schematically in greatlysimplified form, with only those components relevant to understanding ofthe embodiments being illustrated.

Furthermore, the various components (that are identified in the FIG. 1 )are illustrated and the arrangement of the components is presented forpurposes of illustration only. It is to be noted that other arrangementswith more or less components are possible without departing from theteachings of frictionless inquiry processing, presented herein andbelow.

As used herein and below, the terms “customer,” “consumer,” “shopper,”and “user” may be used interchangeably and synonymously.

The system 100 includes a plurality of cameras 110 that capture imagesof persons and items (herein after just “defined area images 111”). Thesystem also includes a server 120, a user-operated device 130 having amobile application (“app”) 131, and one or more inquiry stations 140having an assistance manager 141 and optionally an integrated camera142. The server 120 includes executable instructions that execute on oneor more hardware processors of the server 120 from a non-transitorycomputer-readable storage medium as: an item tracker 121, a persontracker 122, a pose tracker 123, a transaction manager 124, and anassistance manager 125. The user operated device 130 also includes aprocessor and non-transitory computer-readable media having executableinstructions representing the mobile app 131. Similarly, the inquirystation 140 includes a processor and non-transitory computer-readablestorage media having executable instructions representing the assistancemanager 141

It is to be noted that although not illustrated in the FIG. 1 , theserver 120 also includes one or more hardware processors, volatile andnon-volatile memory, non-volatile storage, and networking circuitry(such as wired ports and/or wireless transceivers).

It is also to be noted that there may be multiple servers, such that thedifferent elements 121-124 may execute on a same server 120 or multipledifferent servers networked together.

When a customer enters a store or is outside the store in the parkinglot, cameras 110 begin capturing the time-stamped images 111 in frames.Each image 111 represents a frame in a stream of images. Multiple framesper second are captured by each of the cameras 110.

The cameras 110 are preconfigured to capture images 111 of the definedareas based on the field-of-view of the lenses of the cameras 110. Someof the cameras 110 may capture images 111 representing portions of adifferent area that a different one of the cameras 110 captures images111 for. That is, each image 111 can include pixel values that overlapmultiple ones of the defined areas.

Initially, the cameras 110 are situated in locations throughout anenterprise (such as a retail store but can be other enterprises or evena consumer's home). Each camera lens configured to cover one or morepredefined areas of the physical space of the enterprise.

Furthermore, metadata is assigned to each camera 110 to include a uniquecamera identifier, a location identifier (representing the physicallocation that the camera 110 is situated within the enterprise, and oneor more area identifiers (representing the predefined areas that thelens of the camera 110 captures in the images 111).

Each camera 110 provides time stamp and frame stamped images to theserver 120. These images can be streamed over a wired or wirelessconnection between the cameras 110 and the server 120 to a commonlyaccessible storage area on the server 120 that is accessible to the itemtracker 121, the person tracker 122, and the pose tracker 123. In anembodiment, some of the images when streamed from the cameras 110 can bebuffered or cached in memory of cache and made accessible from thememory or cache to the item tracker 121, the person tracker 122, and thepose tracker 123.

Each accessible image 111 includes its metadata (minimally includingwhat was discussed above) with its image 111 on the server 120.

The person tracker 122 processes the pixels of the images to identify aunique person (the actual identity of the person can be unknown but theperson tracker identifies that a person is in the time-stamped images111). Attributes for the unique person are identified as metadata thatpermit the person tracker 122 to quickly and accurately identify theunique person as that person travels through the store and exits thestore from the time-stamped images 111. Attributes can include clothingtype, color, height, width, shoes, extremity features, eye glasses (sunglasses), hats, eye color, etc. A bounding box is placed around theunique person with the generated metadata. As more images 111 arecaptured from the cameras 110, the additional attributes can be added tothe metadata, some existing attributes can be modified as modifiedmetadata, some existing attributes initially believed to be associatedwith the person can be removed as deleted metadata. The person tracker122 may also have its own machine-learning algorithm that is trainedover time, such that the types of attributes represented in the metadatachanges or the pixel information associated with particular metadata ischanged. In this way, the accuracy of the person tracker 122 improveswith time as does the processing throughput associated with producingthe metadata representing the attributes from the images 111.

In an embodiment, the person tracker 122 is configured with facialrecognition to obtain an identity of a person being tracked from theimages.

In an embodiment, the person tracker 122 is configured for facialrecognition for a unique person present in the store but does not knowan actual identity for the person. That is, the person may beunregistered but still uniquely tracked through unique facial attributesassigned to the unregistered person.

In a similar manner, the item tracker 121 identifies from the images 111items that are handled by the people being tracked by the person tracker122. That is, the item tracker 121 receives the images, crops off pixelsthat are known to not be associated with the item (such as the pixelsassociated with background objects or a person). Each item includes aunique identifier for tracking even though the actual item may beunknown or unrecognized from the images. That is, (and similar to theperson tracker 122), an item identity (such as the item's description,actual item barcode level of detail, etc.) is unknown in thetime-stamped frames but is still assigned and associated with a uniquetracking identifier in the frames/images 111 so as to distinguishbetween other unknown items of the store or other unknown itemspossessed by the customer. Again, attributes associated with the unknownitem is carried as metadata from frame 111 to frame, so that the itemtracker 121 can quickly identify and crop from later-in-time receivedimages 111 the specific pixels or bounding box being tracked for theunknown item. Attributes can include, color, height, width, edges,bottle shape, item label or packaging characteristics, can shape, boxshape, undefined shape, edges, etc. Once enough confidence in themetadata is known for an unknown item, the item is identified andassigned item level details for a known item.

When an individual enters a frictionless store or a traditional storeequipped in some areas with a frictionless store, the person tracker 122interacts with the transaction manager 124 and assigns a transactionidentifier for a transaction with the person. The person tracker 122 isresponsible for tracking the person from frame to frame. The itemtracker 121 identifies items and item identifiers possessed by theperson when held, placed in a bag, or placed in a cart. The itemidentifier is provided to the transaction manager 124. The transactionmanager 124 obtains the item pricing and description using the itemidentifier and adds the item pricing and description to the transactionassociated with the assigned transaction identifier.

The pose tracker 123 utilizes the bounding box and attributes associatedwith the person that are produced and maintained by the person tracker122 to identify a direction or pose of the person in any given frame.The pose tracker 123 is also trained to identify non-item objects thatthe person based on a given pose is believed to be looking at in eachframe. The non-item objects include user-operated electronic devices130, such as a mobile phone, a tablet, and/or a wearable processingdevice (such as a watch). The non-time objects can also include theinquiry stations 140 that have known physical locations within thefrictionless store areas.

Furthermore, the pose tracker 123 utilizes the item tracker 121 toidentify item codes that may be identified in frames associated with theperson indicating that the person is holding an item.

In an embodiment, the pose tracker 123 includes a trainedmachine-learning algorithm that when passed a frame including a personbeing tracked returns an indication as to whether the person is in apose where the person is believed to be looking at a user-operateddevice 130 or an inquiry station 140. If the machine-learning algorithmreturns an identifier associated with an inquiry station or returns anindication that the person is looking at a user-operated device, thepose tracker 120 provides the frame to the item tracker 121, the itemtracker returns an item code if the person is also holding an item andreturns no item code if the person is not holding the time.

When the pose tracker 123 determines that the person is looking at auser-operated device or an inquiry station, the pose tracker providesthe person tracker identifier for the person, any item code for any helditem, and/or an inquiry station identifier for when the person islooking at a given inquiry station 140 to the assistance manager 125.

The assistance manager 125 interacts with the transaction manager 124 todetermine if the person tracker identifier is associated with aregistered and checked-in consumer of the store. When this is the case,the transaction manager returns a consumer identifier. The consumeridentifier is used to access a registered consumer account and obtain amobile device identifier for the user-operated device 130. Thetransaction manager 124 also provides the specific item pricing and itemdescription for any item being held by the registered consumer as wellas item prices and item descriptions for all items that are currentlybelieved to be in the possession of the registered consumer forpurchase. The transaction manager 124 maps the person tracker identifierassigned by the person tracker 122 to the transaction identifierprovided by the transaction manager 124 to the person tracker 122 anddetermines whether the person tracker 122 has identified a registeredconsumer or not based on an automated check in of the consumer (such asby scanning a bar code on entry to the store or through facialrecognition). A profile associated with the registered consumer includesa user-operated device identifier 130 or an identifier for the mobileapp 131. The transaction identifier is maintained for the frictionlesstransaction by the transaction manager.

Assuming the person is a registered consumer with a registereduser-operating device 130, the assistance manager 125 dynamically pushesan application message to the operating system (OS) of the user-operateddevice 130. This forces a wakeup and initiation of the mobile app 131 onthe device 130. The assistance manager 125 then provides the itemdescription and item pricing for any item that the registered consumermay be currently holding when the registered consumer is detected aslooking at the device 130. The complete item prices, item descriptions,and running transaction price total for all the items are also providedin a message sent from the assistance manager 125 to the mobile app 131.

Upon receipt of the messages, the mobile app 131 presents a user-facinginterface on a display of the device 130 to the user and provides withinthat interface a held item price and held item description along with arunning total price for all items in possession of the registeredconsumer. A link associated with the running total price can beactivated by touch in the user-facing interface to display an itemizedlisting of each item description and each item price associated with theongoing transaction. When it is determined that the registered consumeris not holding any item at the time that the consumer looked at thedevice 130, the user-facing interface may present a complete itemlisting for the ongoing transaction along with the item prices and therunning total price. Furthermore, each individual item description mayinclude a link within the user-facing interface that when touched by theregistered consumer displays specific item details, such as size,nutritional information, health warnings if any, and any in-storepromotions associated with the specific item.

If the person is determined to be looking at a specific inquiry station140, the assistance manager 125 sends any held item description andpricing and the transaction item descriptions and prices (along with therunning price total) to the assistance manager 141 of that inquirystation 142. A similar user-facing interface to that which was describedabove for the mobile app 131 is provided by the assistance manager 141on the inquiry station 140.

In an embodiment, the inquiry station 140 may include its own integratedcamera 142, such that when the person looks at the inquiry station 140and is positioned in front of the station 140, the camera 142 capturesthe image and provides to the pose tracker with an indication that theperson is looking at the station 140. The post tracker 123 can theninteract with the item tracker 121, person tracker 122, and transactionmanager 124 using the image to discover the transaction identifier,person identifier, and any item identifier that is being held by theperson. This is provided to the assistance manager 125 or the assistancemanager 141 and the user-facing interface that displays any held itemdescription, any held item description, and running transaction pricetotal is presented. Again, the presented information may includetouchable links that expand out the running transaction price total to afull listing of all item descriptions and prices in the possession ofthe person and item details for each listed item description (as wasdiscussed above). It is also to be noted that the person does not haveto be registered to receive the frictionless inquiry interface providedas the user-facing interface through the assistance manager 141 on thestation 140 because the inquiry station reports a known identifier and atracked person to a transaction identifier does not have to include aregistered consumer to identify how to communicate with the assistancemanager 141. The tracked person can in some instances be a registeredconsumer as was discussed above with the frictionless interface providedas the user-facing interface of the mobile app 131.

The user-facing interface provided by the mobile app 131 and theassistance manager 141 is automatically initiated and activated ondevice 130 and station 140, without any action of the user taking withrespect to the device 130 and station 140. The user-facing interface isautomatically placed in a state that shows information assumed to beingrequested by the user (pricing, descriptions, and details) based on whatif anything the user is holding and the current ongoing frictionlesstransaction being managed by the transaction manager 124.

In an embodiment, the person or registered consumer can pick up an itemafter looking at the device 130 or the station 140 and the frictionlessinterface automatically changes from an itemized presentation for allitems of the transaction to specific item details for the item that waspicked up by the person or registered consumer. In this way, thefrictionless interface can be initiated by looking at device 130 or 140and changed based on an action of the user associated with holding anitem. This is done without the user having to touch the user-facinginterface or interact directly with any links provided in theuser-facing interface.

The system 100 provides a frictionless inquiry interface through theuser-facing interfaces of the mobile app 131 and the assistance manager141. It is frictionless because all the user has to do is look at thedevice 130 or the station 140 and the user is automatically identified(whether registered or not (as can be the case with the station 140),the transaction is identified, any held item is identified, and auser-facing interface with item details or transaction details isautomatically provided through the user-facing interface. All the userhas to do is look at the device 130 or station 140 and the user-facinginterface is initiated and populated with item and transaction detailsassociated with the user.

The transaction manager 124 can check out any given person in a varietyof manners. When the person tracker 122 has identified a customer andthe customer is pre-registered with the store and has a registeredpayment method, then the transaction manager can process that paymentmethod when the person is identified by the person tracker 122 asapproaching or leaving an egress point of the store. When the persontracker 122 was unable to obtain the identity of the person(unregistered), the person can check out at a transaction terminal; thetransaction manager 124 interacts with an agent process on thetransaction terminal 140 and provides the shopping cart items forpayment when the person tracker 122 identifies the person as beingpresent at the transaction terminal. When the person (identified orunidentified) has a mobile application 131 downloaded to the person'suser device 130, the transaction manager interacts with the mobileapplication to present the items in the shopping cart and obtain paymentfrom the person. Other approaches for checkout may be used as well,where the transaction manager 124 and the person tracker 122 cooperateto obtain payment for known persons (identity known) and for unknownpersons (identity unknown).

In an embodiment, the inquiry station 140 is also a transaction terminalfrom which a user/consumer can check-out and pay for items possessed bythe user/consumer. In an embodiment, the transaction terminal is one ormore of: A Point-Of-Sale (POS) terminal and a Self-Service Terminal(SST).

In an embodiment, the inquiry station 140 is a digital sign.

In an embodiment, the inquiry station 140 may also include a bar scannerfrom which consumers can independently scan item barcodes to get itemdetails independent of the frictionless inquiry interface or toindependently verify results of the frictionless inquiry interface.

In an embodiment, the inquiry station 140 is a thin client that lacksthe assistance manager 141 and the frictionless user interface isprovided by the server-based assistance manager 125 on the inquirystation 140.

In an embodiment, the inquiry station 140 lacks any independent camera142.

In an embodiment, the user-operated device 130 is one or more of: aphone, a tablet, a laptop, and a wearable processing device.

These embodiments and other embodiments are now discussed with referenceto the FIGS. 2-4 .

FIG. 2 is a diagram of a method 200 for frictionless inquiry processing,according to an example embodiment. The software module(s) thatimplements the method 200 is referred to as a “frictionless inquirymonitor.” The frictionless inquiry monitor is implemented as executableinstructions programmed and residing within memory and/or anon-transitory computer-readable (processor-readable) storage medium andexecuted by one or more processors of a device. The processor(s) of thedevice that executes the frictionless inquiry monitor are specificallyconfigured and programmed to process the frictionless inquiry monitor.The frictionless inquiry monitor has access to one or more networkconnections during its processing. The network connections can be wired,wireless, or a combination of wired and wireless.

In an embodiment, the device that executes the frictionless inquirymonitor is the server 120. In an embodiment, the server 120 is acloud-based server.

In an embodiment, the frictionless inquiry monitor is all or somecombination of: the item tracker 121, the person tracker 122, the posetracker 123, the assistance manager 125, and/or the assistance manager141.

At 210, the frictionless inquiry monitor identifies within an image aconsumer looking at a device. This can be done through any of themechanisms discussed above with the FIG. 1 including the person tracker122 and the pose tracker 123 and/or including a trained machine-learningalgorithm.

In an embodiment, at 211, the frictionless inquiry monitor identifies adevice identifier for the device based on a registered device associatedwith an account of the consumer. That is, a consumer identifier that isused to track the consumer is linked to the account and the accountincludes a consumer profile having the device identifier.

In an embodiment, at 212, the frictionless inquiry monitor identifiesthe device as a stationary inquiry station 140 located within a storewhere the consumer is shopping. The images include metadata thatidentifies the physical locations represented in the images and thestationary inquiry station includes a known physical location that isassociated with the images.

At 220, the frictionless inquiry monitor causes a user interface to beautomatically initiated on the device based on identifying the consumerlooking at the device as discussed at 210.

In an embodiment, at 221, the frictionless inquiry monitor sends anapplication message to an Operating System (OS) of the device thatinstruction the user interface to be automatically initiated on thedevice. In an embodiment, the user may select a confirmation button thatpops up on the device before the user interface is initiated. In anembodiment, the user interface is the user-facing or frictionlessinterface associated with the mobile application 131 as discussed abovewith the FIG. 1 and the system 100.

In an embodiment, at 222, the frictionless inquiry monitor sends amessage to an inquiry station 140 that instructs the inquiry station 140to automatically initiate the user interface. Here, the device is theinquiry station 140.

At 230, the frictionless inquiry monitor instructs the user interface todisplay on the device information associated with at least one item thatis in a possession of the consumer based on processing the image orprevious images captured for the consumer.

In an embodiment, at 231, the frictionless inquiry monitor provides theinformation to the user interface.

In an embodiment of 231 and at 232, the frictionless inquiry monitorprovides item details and item pricing for an item detected in the imagethat is in a hand of the consumer as the information.

In an embodiment of 232 at 233, the frictionless inquiry monitorprovides a total transaction price and a listing of a plurality of itemsthat are assigned to the consumer for a frictionless store transactionto the user interface with the information.

In an embodiment of 233 at 234, the frictionless inquiry monitorprovides links with the information that when activated from the userinterface by the consumer provides detailed information for the item andeach of the plurality of items.

In an embodiment, at 235, the frictionless inquiry monitor interactswith the user interface to provide different consumer-selected levels ofdetail for a specific item or all items that are in the possession ofthe consumer.

FIG. 3 is a diagram of another method 300 for frictionless inquiryprocessing, according to an example embodiment. The software module(s)that implements the method 300 is referred to as an “automated inquiryservice.” The automated inquiry service is implemented as executableinstructions programmed and residing within memory and/or anon-transitory computer-readable (processor-readable) storage medium andexecuted by one or more processors of a device. The processors thatexecute the automated inquiry service are specifically configured andprogrammed to process the automated inquiry service. The automatedinquiry service has access to one or more network connections during itsprocessing. The network connections can be wired, wireless, or acombination of wired and wireless.

In an embodiment, the device that executes the automated inquiry serviceis the server 120. In an embodiment, the server 120 is a cloudprocessing environment.

In an embodiment, the device that executes the automated inquiry serviceis the station 140.

In an embodiment, the automated inquiry service is all of or somecombination of: the item tracker 121, the person tracker 122, thepose/tracker 123, the assistance manager 125, the assistance manager141, and/or the method 200.

The automated inquiry service presents another and in some ways enhancedprocessing perspective of the method 200 discussed above.

At 310, the automated inquiry service tracks a registered consumerthrough images in a frictionless store.

In an embodiment, at 311, the automated inquiry service obtains metadatafor identifying the registered consumer within the images from a persontracker 122.

At 320, the automated inquiry service detects within the images when theregistered consumer is looking at a device.

In an embodiment, at 321, the automated inquiry service obtains a deviceidentifier for communicating with the device from an account profileassociated with the registered consumer.

In an embodiment, at 322, the automated inquiry service identifies thedevice based on a physical location assigned to the images andassociated with a known inquiry station 140 located at that physicallocation.

At 330, the automated inquiry service instructs the device to initiate auser interface and display on the device transaction informationassociated with a frictionless transaction of the registered consumerwithin the frictionless store.

In an embodiment, at 331, the automated inquiry service obtains thetransaction information from a transaction manager 124 by providing aregistered consumer identifier for the registered consumer to thetransaction manager 124.

In an embodiment, at 332, the automated inquiry service provides aportion of the transaction information as specific item details for aspecific item being held by the registered consumer in the images.

In an embodiment, at 333, the automated inquiry service provides linkswith the transaction information that when activated on the device fromthe user interface by the registered consumer provides varying levels ofdetails with respect to items associated with the frictionlesstransaction.

In an embodiment, at 334, the automated inquiry service provides arunning total price of all items along with item detail for each of theitems to the user interface for display on the device as the transactioninformation.

FIG. 4 is a diagram of a system 400 for frictionless inquiry processing,according to an example embodiment. The system 400 includes a variety ofhardware components and software components. The software components ofthe system 400 are programmed and reside within memory and/or anon-transitory computer-readable medium and execute on one or moreprocessors of the system 400. The system 400 communicates over one ormore networks, which can be wired, wireless, or a combination of wiredand wireless.

In an embodiment, the system 400 implements, inter alia, the processingdescribed above with the FIGS. 1-3 .

The system 400 includes a plurality of cameras 401, a server 402, auser-operated device 406, and an inquiry station 407. The server 402includes at least one hardware processor 403 and configured to executeexecutable instructions from a non-transitory computer-readable storagemedium 404 as frictionless inquiry service 405.

The frictionless inquiry service 405 when executed from thenon-transitory computer-readable storage medium 404 on the processor 403is configured to cause the processor to: identify when a shopper that isshopping in the frictionless store is looking at the user-operateddevice 406 from the images; cause a user interface to automaticallyinitiate on the user-operated device 406 and display transactioninformation associated with a transaction of the shopper; identify whenthe shopper is looking at or in front of the inquiry station 407 basedat least in part on the images; and cause a second user interface toautomatically initiate on the inquiry station 407 and display thetransaction information associated with the transaction of the shopper.

In an embodiment, the frictionless inquiry service 405 when executedfrom the non-transitory computer-readable storage medium 404 on theprocessor 403 is further configured to cause the processor to: providethe transaction information with links that when activated by theshopper from the user interface or the second user interface presentsshopper-controlled details for items associated with the transaction.

In an embodiment, the frictionless inquiry service 405 is all or somecombination of the: image tracker 121, the person tracker 122, the posetracker 123, the transaction manager 124, the assistance manager 125,the method 200, and/or the method 300.

In an embodiment, the server 402 is a cloud-based processingenvironment.

In an embodiment, the user-operated device 406 is: a mobile phone, atablet, or a wearable processing device (such as a smart watch).

In an embodiment, the inquiry station 407 is: a POS terminal, a SST, adigital sign, or a thin client display.

In an embodiment, the system 400 is deployed as a portion of africtionless store implementation where customers (individuals) shopthrough computer-vision and image processing and items and individualsare associated with one another with a shopping cart maintained for eachindividual. Each individual can checkout and pay for his/her shoppingcart items using any of the above-referenced techniques discussed withthe FIG. 1. The frictionless store may be associated with or includedwithin a traditional store as discussed above. That is, a traditionalstore can be equipped with the system 400 in all or a portion of thetraditional store that includes POS terminals and SSTs.

It should be appreciated that where software is described in aparticular form (such as a component or module) this is merely to aidunderstanding and is not intended to limit how software that implementsthose functions may be architected or structured. For example, modulesare illustrated as separate modules, but may be implemented ashomogenous code, as individual components, some, but not all of thesemodules may be combined, or the functions may be implemented in softwarestructured in any other convenient manner.

Furthermore, although the software modules are illustrated as executingon one piece of hardware, the software may be distributed over multipleprocessors or in any other convenient manner.

The above description is illustrative, and not restrictive. Many otherembodiments will be apparent to those of skill in the art upon reviewingthe above description. The scope of embodiments should therefore bedetermined with reference to the appended claims, along with the fullscope of equivalents to which such claims are entitled.

In the foregoing description of the embodiments, various features aregrouped together in a single embodiment for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting that the claimed embodiments have more features than areexpressly recited in each claim. Rather, as the following claimsreflect, inventive subject matter lies in less than all features of asingle disclosed embodiment. Thus the following claims are herebyincorporated into the Description of the Embodiments, with each claimstanding on its own as a separate exemplary embodiment.

The invention claimed is:
 1. A method, comprising: tracking a consumerwithin a store from images captured by cameras of the store; monitoringthe consumer and determining the consumer is looking at a display of adevice, wherein monitoring further includes identifying an item beingheld by the consumer from the images as the consumer looks at a displayof the device; and initiating an interface on a display of the devicebased on the determining without any interaction of the consumer withthe interface or the device that the consumer is looking at the displaywhile holding the item.
 2. The method of claim 1, wherein trackingfurther includes identifying the consumer as a registered consumer fromthe images.
 3. The method of claim 2, wherein identifying furtherincludes obtaining an account profile associated with the registeredconsumer.
 4. The method of claim 3, wherein obtaining further includesobtaining a device identifier for the device from the account profile.5. The method of claim 4, wherein monitoring further includes using thedevice identifier to communicate with the device.
 6. The method of claim5, wherein using further includes rendering the interface on the device.7. The method of claim 1, wherein tracking further includes linking theconsumer to a shopping cart being maintained for the consumer within thestore as the consumer picks items for purchase.
 8. The method of claim7, wherein initiating further includes presenting transaction detailsfor the shopping cart within the interface on the display of the device.9. The method of claim 1, wherein initiating further includes presentingitem details for the item within the interface on the display of thedevice.
 10. A method, comprising: monitoring a consumer shopping withina store from images captured by cameras of the store; maintainingtransaction information for a transaction of the consumer based on itemsplaced by the consumer in a bag or cart and identified from the images;determining that the consumer is holding a particular item; determiningthat consumer is looking at a display of a device; identifying thedevice and obtaining a device identifier for the device; and presentingat least a portion of the transaction information on a display of thedevice using the device identifier based on the determining that theconsumer is holding the particular item and is looking at the displaywithout any interaction of the consumer with an interface of the device.11. The method of claim 10, wherein identifying further includesidentifying the device from the images as a consumer device registeredto the consumer.
 12. The method of claim 10, wherein identifying furtherincludes identifying the device based on a current location calculatedfrom the consumer within the store from the images.
 13. The method ofclaim 10, wherein identifying further includes identifying the device asa stationary inquiry station located within the store that the consumeris standing in front of based on the images.
 14. The method of claim 10,wherein identifying further includes identifying the device as atransaction terminal of the store where the consumer is ready to checkout for the transaction.
 15. The method of claim 10, wherein presentingfurther includes initiating a user interface on the device andpresenting a summary of the transaction information within the userinterface.
 16. The method of claim 10, wherein presenting furtherincludes initiating a user interface on the device and presentingparticular item information for the particular item that the consumer isidentified as holding from the images within the user interface.
 17. Themethod of claim 16, wherein initiating further includes presentingdetailed transaction information for the items in the bag or the cartwithin the user interface for the transaction based on user interactionwith the user interface.
 18. A system, comprising: cameras situatedthrough a store to capture images; a server comprising a processor and anon-transitory computer-readable storage medium; the non-transitorycomputer-readable storage medium comprises executable instructions; andthe executable instructions when executed by the processor from thenon-transitory computer-readable storage medium cause the processor toperform operations, comprising: tracking and monitoring a consumerwithin the store through the images; maintaining item information foritems in possession of the consumer while within the store as atransaction of the consumer with the store based on the tracking;determining when the consumer is looking at a display of a device whilethe consumer is holding a particular item based on the tracking;identifying the device and obtaining a device identifier for the device;and presenting the item information corresponding to the particular itemon the display of the device using the device identifier and based onthe determining without any interaction by the consumer with aninterface of the device based on a determination that the customer islooking at the display of the device while holding the particular item.19. The system of claim 18, wherein the device is a phone of theconsumer, a watch of the consumer, a tablet of the consumer, atransaction terminal of the store, or an inquiry station of the store.